Scoring Structural Mde: towards More Meaningful Error Rates
نویسندگان
چکیده
This paper outlines issues and explores options for augmenting the current structural MDE scoring with the goal of obtaining more meaningful measures of performance. We propose event-based statistical significance measures (similar to STT), evaluation of confidence predictions (important to downstream processing applications), and accounting for annotation differences (due to inherent variability in annotations). Focusing on SUs, we give results for SRI-ICSI-UW system outputs for reference and STT conditions with preliminary implementations of these proposals.
منابع مشابه
Phylogeny can make the mid-domain effect an inappropriate null model.
The mid-domain effect (MDE), a bias in species richness towards the midpoint of a given geographical dimension, has been used as a null model in macro-ecological studies. Departures from a MDE are often thought of as interesting. The MDE is a product of the interaction between geometric boundaries and range locations, with species being forced to occupy more central positions in proportion to t...
متن کاملStructural metadata annotation: moving beyond English
The goal of metadata extraction (MDE) is to enable technology that can take raw speech-to-text output and refine it into forms that are more useful to humans and to downstream automatic processes. Starting in 2003, a structural metadata annotation task was defined for English as part of the DARPA EARS Program. A significant new challenge for MDE is the addition of new languages. This paper repo...
متن کاملEXPLORING EFFICIENT AND SCALABLE OMNISCIENT DEBUGGING FOR MDE by JONATHAN CORLEY JEFF GRAY, COMMITTEE CHAIR
Model-Driven Engineering (MDE) has emerged as a software development paradigm that can assist in separating the issues of the problem space of a software system from a particular solution space of implementation. MDE approaches often use customized domain-specific modeling languages that capture the intent of a particular group of end users through abstractions and notations that fit a specific...
متن کاملEXPLORING EFFICIENT AND SCALABLE OMNISCIENT DEBUGGING FOR MDE by JONATHAN CORLEY
Model-Driven Engineering (MDE) has emerged as a software development paradigm that can assist in separating the issues of the problem space of a software system from a particular solution space of implementation. MDE approaches often use customized domain-specific modeling languages that capture the intent of a particular group of end users through abstractions and notations that fit a specific...
متن کاملTowards optimal alignment of protein structure distance matrices
MOTIVATION Structural alignments of proteins are important for identification of structural similarities, homology detection and functional annotation. The structural alignment problem is well studied and computationally difficult. Many different scoring schemes for structural similarity as well as many algorithms for finding high-scoring alignments have been proposed. Algorithms using contact ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004